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Abstract-  This  paper  reports  on  a  methodology  developed  for 
the  analysis  of  near-bottom  photographs  collected  for  fisheries 
habitat  studies.  These  tools  provide  a  framework  for  conducting 
minimally  invasive  in-situ  investigations  of  benthic  organism 
abundance,  diversity,  and  distribution  using  high-resolution 
optical  datasets  integrated  with  high  precision  navigational  data. 
Utilizing  these  techniques  with  near-bottom  photos  collected  with 
a  precision  navigated  survey  platform  greatly  increases  the 
efficiency  of  image  analysis  and  provides  new  insight  about  the 
relationships  between  benthic  organisms  and  the  habitats  in 
which  they  are  found. 

Basic  requirements  for  the  analysis  of  near-bottom  seafloor 
images  include  camera  calibration  and  quantification  of  the 
height  of  the  lens  above  the  seafloor  throughout  the  survey. 
Corrections  are  required  to  compensate  for  image  distortion  due 
to  lighting  limitations  and  the  variable  micro-topography  of  the 
seafloor.  These  parameters  can  be  constrained  by  utilizing 
precisely  navigated  survey  platforms  such  as  Autonomous 
Underwater  Vehicles  (AUVs)  or  Remote  Operated  Vehicles 
(ROVs). 

The  methodology  we  present  was  developed  with  data 
collected  by  the  SeaBED  AUV  off  the  coast  of  Washington, 
Oregon  and  California  [1],  A  digital  database  containing  benthic 
organism  identifications,  measurements,  and  locations  was 
generated  for  each  image  using  a  Graphical  User  Interface  (GUI) 
created  in  Matlab™  [1,2].  This  methodology  has  demonstrated 
a  significant  increase  in  the  efficiency  of  image  analysis  for 
benthic  habitat  studies,  and  provides  the  opportunity  to  assess 
small  scale  spatial  distribution  of  organisms  in  their  natural 
habitats.  Collecting  overlapping  images  permits  the  creation  of 
photomosaics  [3]  and  the  quantification  of  organism  abundance 
per  unit  area  of  the  seafloor. 


I.  Introduction 

Fish  population  assessments  have  traditionally  been  based 
upon  observations  of  organisms  captured  in  fishing  trawls 
Although  observations  are  not  made  in-situ,  correlations  can 
be  made  between  fish  populations  and  observations  of  seabed 
characteristics  derived  from  co-located  sonar  surveys  and/or 
bottom  photographs  collected  in  discrete  locations.  Fish 
populations  that  dwell  in  rocky  environments,  however,  are 
difficult  to  assess  with  trawl  surveys  because  of  the  risk  of 
fowling  gear  on  the  seafloor.  Surveys  have  also  been 
conducted  with  video  cameras  placed  on  towed  vehicles  but 
these  platforms  are  also  not  ideal  for  rugged,  rocky  habitats. 


Autonomous  Underwater  Vehicles  (AUVs)  with  precise 
navigation  provide  new  alternatives  to  fish  population  and 
habitat  assessment.  Near-bottom  high-resolution  digital 
photographic  surveys  conducted  with  AUVs  are  minimally 
invasive,  and  provide  a  means  for  in-situ  observations  of 
organisms.  When  performed  at  slow  survey  speeds,  a 
continuum  of  photographic  information  can  be  acquired  along 
the  survey  track.  These  images  not  only  provide  quantitative 
data  about  benthic  habitat,  but  can  also  be  used  to  identify  fish 
in  their  natural  habitat.  Surveys  of  this  nature  also  provide  a 
means  for  ground-truthing  geoacoustic  facies  identified  with 
high-resolution  geophysical  mapping  techniques. 

Since  hundreds  of  digital  images  are  typically  collected 
during  each  AUV  dive,  analysis  can  be  very  time  consuming. 
Software  tools  that  allow  scientists  to  digitally  identify  and 
annotate  organisms  imaged  in  photos  are  required  to 
efficiently  utilize  these  vast  datasets.  When  in-situ 
observations  of  organisms  are  combined  with  precision 
navigational  data,  new  insights  can  be  gained  about  the  spatial 
distribution  of  organisms  and  the  habitats  in  which  they  live. 

II.  Data  Acquisition 


Figure  1.  The  SeaBED  Autonomous  Underwater  Vehicle  (AUV)  is 
designed  for  benthic  studies  in  rugged  terrain.  It  can  be  used  to  conduct  low- 
altitude  (2.5  m)  surveys  yielding  high-resolution  digital  images. 
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The  SeaBED  AUV  (Fig.  1)  is  navigated  through  the  use  of  a 
Doppler  Velocity  Log  (DVL)  navigation  system,  resulting  in 
meter-scale  navigational  precision  [4,5].  It  is  equipped  with  a 
high-dynamic  range  down-looking  camera  that  collects  high- 
resolution  (0.15  cm/pixel)  color  imagery. 

The  nonlinear  attenuation  of  visible  light  in  seawater  means 
that  images  typically  wind  up  looking  far  more  green  or  blue 
as  we  move  away  from  the  object  being  images.  That  is,  the 
reds  in  the  imagery  are  preferentially  attenuated  as  a  function 
of  distance.  If  we  use  standard  (8  bit)  dynamic  range  cameras 
the  level  of  signal  in  the  red  channel  in  the  imagery  is  very 
small,  comparable  to  the  noise  levels  of  the  camera.  However, 
by  utilizing  high  dynamic  range  12  bit  imagery,  even  though 
the  signal  level  of  the  red  channel  is  small,  it  is  enough  that  we 
can  rescale  it  to  compensate  for  the  nonlinear  attenuation. 

The  rescaling  process  for  the  different  color  channels  is 
carried  out  by  fitting  a  polynomial  surface  to  each  of  RGB 
channels  of  the  image  to  bring  them  into  rough  parity  in  terms 
of  signal  strength.  By  utilizing  a  single  ground  truth  point  in 
color  space  the  resulting  equalized  RGB  imagery  may  be 
transformed  to  obtain  colors  that  are  representative  of  what  we 
would  see  if  we  could  image  the  scene  without  water  in 
sunlight.  Figure  2  shows  an  example  of  SeaBED  imagery 
before  and  after  color  correction. 


Figure  2.  Typical  12  bit  dynamic  range  imagery  collected  by  the  SeaBED 
AUV  (top)  and  the  results  after  color  correction  (bottom). 

All  our  imagery  is  color  corrected  in  this  fashion.  The 
imagery  is  also  post-processed  to  compensate  for  lens 
distortion. 

Over  30,000  near-bottom  digital  images  were  acquired  with 
SeaBED  during  a  2  week  cruise  in  October  2006  aboard  the 
R/V  Thomas  G.  Thompson.  AUV  surveys  complemented 
seafloor  mapping  conducted  at  three  fishing  areas  along  the 


west  coast  of  the  U.S.  from  Washington  to  California.  AUV 
surveys  were  conducted  at  an  altitude  of  2.5  m  yielding  an 
average  image  area  of  3  m2. 

Although  we  focus  in  this  paper  on  the  imagery  collected 
during  this  expedition,  the  tools  reported  here  are  equally 
applicable  to  imagery  collected  from  other  imaging  platforms. 

A  Graphical  User  Interface  (GUI)  was  developed  in 
Matlab™  for  identification  and  digital  annotation  of 
organisms  visible  in  the  images.  The  GUI  is  used  to  record  a 
list  of  organisms,  information  about  their  size  and  habitat  type, 
and  the  geospatial  information  associated  with  each  image. 

III.  Graphical  User  Interface 

The  interface  that  was  developed  merges  digital 
photographs  with  the  vehicle’s  precision  navigation.  It  is  used 
to  browse  images  and  to  identify  and  annotate  organisms  and 
objects  evident  in  them  (Fig.  3).  Navigational  data  are  used  to 
reference  the  images  geographically,  and  altitude  data  are  used 
to  estimate  the  image  footprint  assuming  a  flat  seafloor. 

Organism  taxa  are  chosen  from  predetermined  lists,  and 
each  individual  is  identified  and  counted  with  a  click  of  the 
mouse  button.  For  each  organism  that  is  identified  a  cross¬ 
hair  symbol  is  plotted  based  on  a  user  defined  position  in  the 
image.  Organisms  not  included  in  the  lists  can  be  counted  and 
identified  via  manual  entry  of  taxa  information.  Length  and 
area  tools  are  provided  to  assign  measurements  to  organisms 
and/or  seafloor  objects,  based  on  the  assumption  that  objects 
are  on  the  seafloor.  Benthic  habitats  are  described  for  each 
image  based  on  a  binary  classification  system  for  describing 
sediment  type  (Fig.  4). 

Data  tabulated  for  each  image  include  navigation,  attitude, 
estimated  image  area,  organism  identifications,  positions 
within  the  image,  and  substrate  type  are  logged  in  a  comma 
delimited  text  file.  When  the  GUI  is  reopened  and  images 
previously  analyzed  are  viewed,  previously  defined 
annotations  are  displayed  on  the  image.  This  provides  a 
means  for  quality  control  and  for  adding  additional 
annotations.  Upon  closing  the  interface,  all  text  files 
contained  within  the  working  directory  are  assembled  into  a 
self-updating  master  list.  An  option  is  also  provided  to  save 
annotated  tiff  images. 


Figure  3.  FI  SIT  ROCK  interface  for  computer -assisted  identification  of 
benthic  organisms. 
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Figure  5.  Average  number  of  rockfish  per  image  based  on  substrate  type. 


>25.5  cm _ 25.5 -6.4  cm  <6.4 -0.2  cm  <2 -0.06  mm  <.06  mm 

decreasing  particle  size 


Figure  4.  Binary  habitat  classification  scheme  (after  [6]).  The  first  letter  in 
the  binary  scheme  represents  the  dominant  substrate  (>  50%  image  area), 
while  the  second  letter  represents  the  secondary  substrate  (>  20%  image  area). 


Figure  6.  Average  number  of  flatfish  per  image  based  on  substrate  type. 


Data  generated  with  the  GUI  were  also  imported  directly 
into  Geographic  Information  System  (GIS)  software  to 
investigate  the  spatial  distribution  of  benthic  organisms  (Fig. 
7).  This  reveals  relationships  between  benthic  organisms  and 
seafloor  characteristics  derived  from  geophysical  mapping 
techniques.  In  addition,  each  observation  plotted  in  Fig.  6  can 
be  directly  linked  to  the  original  image  of  the  seafloor. 


IV.  Application  of  Interface 

Three  different  sample  designs  were  tested  to  determine  the 
optimal  number  of  transects  and  images  required  for 
statistically  significant  results.  Two  of  these  sample  designs 
utilize  alternate  images  along  a  transect,  while  the  third  design 
uses  random  images.  The  random  image  approach  requires 
that  an  order  of  magnitude  fewer  images  be  analyzed  than  the 
other  approaches. 

Observations  of  benthic  organisms  were  used  to  investigate 
relationships  between  benthic  habitat  and  two  dominant 
categories  of  fishes:  Rockfish  and  Flatfish.  Roughly  half  of 
all  organisms  identified  in  each  category  were  identified  to 
species  (46%  and  65%  respectively).  As  expected,  the 
majority  of  rockfish  were  identified  in  habitats  dominated  by 
rocks,  boulders  and  cobble  (Fig.  5).  Similarly,  the  majority  of 
flatfish  were  identified  in  boulder-dominated  habitats  (Fig.  6). 


1  Fish  ID 

• 

bigfin  eel  pout 

• 

hagfish 

• 

poacher/cottid 

• 

rosethorn  rockfish 

<§> 

splitnose  rockfish 

• 

splitnose/aurora 

spotted  ratfish 

thornyhead  unident 

• 

threadfin  sculpin 

• 

unknown  eelpout 

• 

unknown  poacher 

• 

unknown  rockfish 

Figure  7.  Identified  fish  overlain  on  bathymetry  data  in  GIS. 


V.  Conclusions  and  plans  for  Future  Development 

Precision  navigated  vehicles  equipped  with  high-resolution 
digital  cameras  are  becoming  increasingly  common  tools  used 
for  a  variety  of  oceanographic  investigations.  The  computer- 
assisted  approach  to  analyzing  digital  photos  described  here 
provides  a  framework  for  quantitatively  using  these  datasets 
for  in  situ  studies  of  benthic  organisms  in  their  natural  habitats. 
Although  developed  for  an  AUV,  this  approach  can  also  be 
used  with  precisely  navigated  towed  camera  systems. 

Further  development  of  this  software  will  utilize  feature 
matching  functionality  used  in  photo-mosaicing  software  [3] 
to  automatically  remove  observations  of  organisms  identified 
in  consecutive  images  (Fig.  8).  This  will  result  in  a  continuum 
of  observations  along  the  entire  AUV  survey  track,  and  will 
permit  the  calculation  of  organisms  per  unit  area.  This 
functionality  has  already  been  developed,  and  it  will  soon  be 
implemented.  More  complex  algorithms  will  also  be  included 
to  perform  supervised  classification  of  organisms.  In  addition, 
consecutive  images  with  overlap  can  be  used  for  3D  image 
reconstruction.  This  could  be  used  to  quantify  seafloor 
microtopography,  and  to  measure  objects  and  organisms  that 
are  not  on  the  seafloor. 


Figure  8.  Photo-mosaicing  algorithm  combined  with  observations  of 
organisms.  This  technique  can  be  used  to  remove  redundant  data  from 
consecutive  images. 
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